…Entity Store maintainer (#259732)

## Summary

Implements the v2 risk scoring pipeline — a new ES|QL-based maintainer
that scores entities by EUID, reads modifiers directly from the Entity
Store, uses run-aware reset logic, and resolves entity groups via a
lookup index. All new behaviour is feature-flagged and runs alongside
the existing risk engine without affecting it, including the preview
risk scores API.

Foundation work was merged separately in
[#258197](#258197) (maintainer
registration, legacy API guards, saved object setup).

---

### What changed from v1

| Concern | v1 (legacy risk engine) | v2 (risk score maintainer) |
|---|---|---|
| **Entity identity** | `user.name` / `host.name` | EUID (`host:<id>`,
`user:<email@namespace>`) via `getEuidEsqlEvaluation()` |
| **Scoring engine** | Composite agg → ES\|QL per page | Same two-step
pattern, but the ES\|QL query now computes EUIDs inline and scores by
`entity.id` |
| **Modifier source** | Separate queries to asset criticality index +
privmon index | Read directly from pre-fetched Entity Store documents
(`entity.asset.criticality`, `entity.attributes.watchlists`) |
| **Watchlist model** | Hardcoded `is_privileged_user` boolean | Generic
watchlist system — any number of named watchlists with configurable
`riskModifier` values, applied multiplicatively |
| **Resolution scoring** | Not supported | Phase 2 — entities sharing a
resolution target are grouped and scored as one via `LOOKUP JOIN` (see
below) |
| **Persistence** | Risk index only | Dual-write: risk index + Entity
Store (`entity.risk.*` for base,
`entity.relationships.resolution.risk.*` for resolution) |
| **Shadow entities** | All scored entities are written | Entities not
in the Entity Store are intentionally dropped from persistence |
| **Entity categorization** | None | Each scored entity is classified as
`write_now`, `defer_to_phase_2`, or `not_in_store` to route between
phases |
| **Reset mechanism** | Per-page exclusion list (`excludedEntities`) |
Run-aware stale detection via `calculation_run_id` — dual pass for
`base` and `resolution` score types |
| **Orchestration** | `risk_score_service` → transforms |
`risk_score_maintainer` registered with the Entity Store maintainer
framework |
| **Preview API** | Full v1 scoring pipeline (stateful) | V2 preview
returns base scores only with `entity.id`-based identifiers; resolution
scoring and reset-to-zero are excluded as preview is stateless |

### How reset-to-zero changed

v1 accumulates a global `scoredEntityIds` array across all pages, then
resets any entity with a positive score whose ID is not in that array.
This requires unbounded in-memory state and breaks down in a multi-phase
pipeline where later phases write additional score documents after base
pagination completes.

v2 replaces this with run-aware stale detection:

1. Query the risk index for each entity's **latest** score doc (`STATS
... LAST(score, @timestamp) BY id_value`)
2. Scope to the target score type (`WHERE score_type IS NULL OR
score_type == "<type>"`) — runs independently for `base` and
`resolution`
3. Reset only if the latest positive score was **not** produced by the
current run (`calculation_run_id IS NULL OR calculation_run_id !=
currentRunId`)
4. Bounded to `LIMIT 10000` per run — remaining stale entities are
handled by subsequent scheduled runs

This removes the need for per-page exclusion state and composes cleanly
with multiple score types.

<details>
<summary>Reset ES|QL query</summary>

```sql
FROM {risk_score_alias}
  | EVAL id_value = TO_STRING({entityField})
  | EVAL score = TO_DOUBLE({scoreField})
  | EVAL score_type = TO_STRING({scoreTypeField})
  | EVAL calculation_run_id = TO_STRING({runIdField})
  | WHERE id_value IS NOT NULL AND id_value != ""
  | WHERE score_type IS NULL OR score_type == "{targetScoreType}"
  | STATS
      score = LAST(score, @timestamp),
      calculation_run_id = LAST(calculation_run_id, @timestamp)
    BY id_value
  | WHERE score > 0
  | WHERE calculation_run_id IS NULL OR calculation_run_id != "{currentRunId}"
  | KEEP id_value
  | LIMIT 10000
```

</details>

### Schema updates

Two new fields are added to the risk score index mapping. Both are
optional and backward-compatible — existing v1 documents without these
fields continue to work.

<details>
<summary>New risk score document fields</summary>

| Field | Type | Purpose |
|---|---|---|
| `score_type` | `keyword` | Classifies the score document: `base`,
`propagated`, or `resolution`. Absent on v1 documents. |
| `calculation_run_id` | `keyword` | UUID of the maintainer run that
produced the document. Used for run-aware stale detection in reset. |
| `related_entities` | array of `{ entity_id, relationship_type }` |
Present on resolution score documents — lists the contributing aliases
in the resolution group. |

</details>

<details>
<summary>Modifier type changes</summary>

The `Modifier<'watchlist'>` type was generalized:

```typescript
// Before (v1)
{ type: 'watchlist', subtype: 'privmon', metadata: { is_privileged_user: boolean } }

// After (v2)
{ type: 'watchlist', subtype: string, metadata: { watchlist_id: string } }
```

Multiple watchlist modifiers per entity are supported and combine
multiplicatively. The `riskScoreDocFactory()` third parameter changed
from `Modifier<'watchlist'> | undefined` to
`Array<Modifier<'watchlist'>>`. Backward compatibility with the existing
privmon modifier is preserved.

</details>

### Pipeline walkthrough

The maintainer `run()` executes per entity type (host, user, etc.):

**Phase 1 — Base scoring**

1. **Fetch watchlist configs** once per run via `WatchlistConfigClient`
2. **Paginate entity IDs** — composite aggregation with a Painless
runtime mapping that computes EUIDs
3. **Score the page** — ES|QL query bounded to the page's EUID range,
using `MV_PSERIES_WEIGHTED_SUM(TOP(risk_score, sampleSize, "desc"),
1.5)`
4. **Fetch entities** from Entity Store via `crudClient.listEntities()`
for the scored page
5. **Apply modifiers** — `applyScoreModifiersFromEntities()` reads
criticality + watchlists from entity documents
6. **Categorize & persist** — entities classified into `write_now` /
`defer_to_phase_2` / `not_in_store`; dual-write to risk index + Entity
Store
7. **Sync lookup index** — upsert alias→target relationship docs for
entities with resolution relationships into
`.entity_analytics.risk_score.lookup-{namespace}` (indexed with
`index.mode: lookup`)

**Phase 2 — Resolution scoring**

8. **Paginate resolution groups** — composite aggregation over
`resolution_target_id` in the lookup index
9. **Score each group** — ES|QL query with `LOOKUP JOIN` joins the
lookup index to combine alerts from all aliases in the group
10. **Parse related entities** — contributing aliases extracted as
`related_entities` from the join result
11. **Apply group-level modifiers** — highest asset criticality across
the group, union of all watchlists
12. **Persist** — resolution scores written to risk index (`score_type:
"resolution"`) and Entity Store
(`entity.relationships.resolution.risk.*`)

**Cleanup**

13. **Reset to zero** — run-aware stale detection clears decayed scores
(dual pass for `base` and `resolution`)
14. **Prune lookup index** — removes stale relationship docs for
entities no longer in the Entity Store

**Preview API (synchronous)**

The existing `/internal/risk_score/preview` endpoint is extended with a
V2 code path, gated by the `entityAnalyticsEntityStoreV2` experimental
feature flag. When V2 is active:

- Runs the same ES|QL base scoring query as Phase 1, but synchronously
within the HTTP request
- Returns `id_field: "entity.id"` and raw EUIDs as `id_value` (e.g.
`host:<id>`, `user:<email>`) — no translation to legacy identifier
fields
- Applies Entity Store modifiers (asset criticality + watchlists) when
entities are enrolled
- Omits resolution scoring and reset-to-zero, as preview is stateless
and read-only
- Falls back transparently to the V1 scoring path when the flag is
disabled

### Telemetry

New event-based telemetry for observability:

- **`risk_score_maintainer_run_summary`** — one per `{namespace,
entityType, calculationRunId}` with outcome, counters, and duration
- **`risk_score_maintainer_stage_summary`** — one per stage
(`phase1_base_scoring`, `phase1_lookup_sync`,
`phase2_resolution_scoring`, `reset_to_zero`) with per-stage timing and
error details
- Run counters include: `scoresWrittenBase`, `scoresWrittenResolution`,
`lookupDocsUpserted`, `lookupDocsDeleted`, `lookupPrunedDocs`
- Bounded taxonomies for status (`success | error | skipped | aborted`),
skip reasons, and error kinds

---

## Testing

### Feature flags

Add to `kibana.yml` (or `kibana.dev.yml`):

```yaml
xpack.securitySolution.enableExperimental:
  - entityAnalyticsEntityStoreV2
  - entityAnalyticsWatchlistEnabled
```

Then enable Entity Store V2 and the ID-based risk scoring advanced
setting (`securitySolution:entityStoreEnableV2`) via Stack Management →
Advanced Settings.

### Manual testing with the document generator

The
[security-documents-generator](https://github.com/elastic/security-documents-generator)
has a companion PR
([#342](elastic/security-documents-generator#342))
with a `risk-score-v2` command that seeds entities, ingests alerts,
triggers maintainer runs, and prints a scorecard:

```bash
# In the security-documents-generator repo, on the risk-scoring-v2 branch:
yarn start risk-score-v2
```

The command supports interactive follow-on actions (expand entities,
tweak scores, view risk docs, run comparisons) and includes graph
workflows for resolution + propagation.

### What to verify

- Entities enrolled in the Entity Store receive risk scores; entities
not in the store do not
- Scores appear in both the risk index (`.ds-risk-score.risk-score-*`)
and on Entity Store documents (`entity.risk.*`)
- `score_type: "base"` and `calculation_run_id` are present on new risk
score documents
- Asset criticality modifiers are applied correctly (sourced from entity
documents, not the criticality index)
- Watchlist modifiers apply multiplicatively when an entity is on
multiple watchlists
- Entities sharing a resolution target produce a single resolution score
with `score_type: "resolution"` and `related_entities`
- Resolution scores carry the highest criticality and union of
watchlists across the group
- Aliases in a resolution group do not receive their own resolution
score
- Resolution scores are dual-written to Entity Store under
`entity.relationships.resolution.risk.*`
- When alerts age out of the risk window, scores decay to zero on the
next maintainer run
- Legacy risk engine routes return 400 when V2 is enabled
- Disabling the feature flags restores normal v1 behavior with no side
effects
- **Preview API returns V2 base scores with `entity.id`-based
identifiers when V2 is enabled, and falls back to V1 scoring when
disabled**

### FTR test coverage

```
node scripts/jest_integration \
  --config x-pack/solutions/security/test/security_solution_api_integration/test_suites/entity_analytics/risk_score_maintainer/trial_license_complete_tier/configs/ess.config.ts
```

| Suite | Covers |
|---|---|
| `setup_and_status.ts` | Maintainer registration, index template
creation |
| `task_execution.ts` | Scoring, restart, manual run, asset criticality
integration |
| `task_execution_nondefault_spaces.ts` | Namespace-scoped scoring
isolation |
| `risk_score_calculation.ts` | Single/multi-entity scoring, criticality
modifiers, watchlist modifiers |
| `resolution_scoring.ts` | Resolution group aggregation, multi-alias
groups, group-level modifiers, Entity Store dual-write |
| `preview_api.ts` | V2 base-only preview scores, entity.id-based
identifiers, Entity Store modifier application, V1 fallback |

---

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

### Risks

| Risk | Severity | Mitigation |
|---|---|---|
| New maintainer scoring diverges from v1 in edge cases | Medium | FTR
tests assert parity; v2 is fully feature-flagged and does not affect v1
code paths |
| Dual-write to Entity Store adds write load | Low | Writes are
conditional on `idBasedRiskScoringEnabled` and errors are non-fatal
(logged as warnings) |
| Reset-to-zero batch limit (10k) may leave stale scores temporarily |
Low | By design — remaining stale entities are cleared in subsequent
scheduled runs |
| Lookup index grows with entity relationships | Low | Pruned each
maintainer run; index uses `index.mode: lookup` for efficient joins |
| Resolution scoring adds a second pagination pass | Low | Only runs
when resolution relationships exist; bounded by the same composite
pagination limits |

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: kubasobon <kuba.sobon@elastic.co>
Co-authored-by: Kuba Soboń <wtty.fool@gmail.com>
Co-authored-by: Tiago Vila Verde <tiago.vilaverde@elastic.co>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: CAWilson94 <charlotte.wilson@elastic.co>

…d risk contributions (#260984)

## Summary

Add the resolution score to the entity flyout and the risk score
contributions.

<img width="3009" height="1575" alt="Screenshot 2026-04-02 at 17 36 57"
src="https://github.com/user-attachments/assets/5437d8a3-4f63-4d39-8aa8-4915bc9cff91"
/>

- Updates the entity flyout risk summary to show entity and
resolution-group scoring coherently, including corrected score sourcing
and filtering behavior.
- Reworks the risk contributions tab for resolution scoring by replacing
the separate contributing-entities section with inline attribution in
alerts and contexts.
- Uses entity-store-driven attribution for resolution context rows
(watchlists and criticality) and alert-derived attribution for
contributing alerts.
- Reduces flyout refresh noise by limiting loading indicators to
initial/empty states and adjusting table layout for readability.
- Changes attribution display to Entity ID and links host/user IDs to
preview-mode entity flyouts.

## Testing

### Feature flags

Add to `kibana.yml` (or `kibana.dev.yml`):

```yaml
xpack.securitySolution.enableExperimental:
  - entityAnalyticsEntityStoreV2
  - entityAnalyticsWatchlistEnabled
```

Then enable Entity Store V2 and the ID-based risk scoring advanced
setting (`securitySolution:entityStoreEnableV2`) via Stack Management ->
Advanced Settings.

### Manual testing with the document generator

The
[security-documents-generator](https://github.com/elastic/security-documents-generator)
has a companion PR
([#342](elastic/security-documents-generator#342))
with a `risk-score-v2` command that seeds entities, ingests alerts,
triggers maintainer runs, and prints a scorecard:

```bash
# In the security-documents-generator repo, on the risk-scoring-v2 branch:
yarn start risk-score-v2
```

The command supports interactive follow-on actions (expand entities,
tweak scores, view risk docs, run comparisons) and includes graph
workflows for resolution + propagation.

### What to verify

- In the entity flyout risk summary, the top card/table show the entity
(non-resolution) risk score and the card label reads `Entity risk score`
- The resolution card/table appears only when a resolution score exists
and is labeled `Resolution group risk score`
- Resolution score history is loaded using the resolution target
`id_value` and `score_type: "resolution"`
- In the Risk contributions tab, the entity/resolution toggle appears
only when a resolution score exists
- The separate contributing-entities section is removed; attribution is
shown inline in table columns
- In resolution view, both Contexts and Alerts tables show an `Entity
ID` column
- For context rows, watchlist and criticality attribution can show
multiple entity IDs when multiple entities match
- Watchlist rows display watchlist names (resolved from watchlist API
data), not only IDs
- Host/user entity IDs in attribution columns open preview-mode entity
flyouts (without replacing the current flyout)
- During background refreshes, existing table rows stay visible and
loading indicators appear only for initial/empty states

### Automated checks run

- `node scripts/jest
x-pack/solutions/security/plugins/security_solution/public/entity_analytics/components/entity_details_flyout/tabs/risk_inputs/risk_inputs.test.tsx`
- `node scripts/check_changes.ts`

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: kubasobon <kuba.sobon@elastic.co>
Co-authored-by: Kuba Soboń <wtty.fool@gmail.com>
Co-authored-by: Tiago Vila Verde <tiago.vilaverde@elastic.co>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: CAWilson94 <charlotte.wilson@elastic.co>
Co-authored-by: jaredburgettelastic <jared.burgett@elastic.co>
Co-authored-by: Jared Burgett <147995946+jaredburgettelastic@users.noreply.github.com>

…d risk contributions (elastic#260984)

## Summary

Add the resolution score to the entity flyout and the risk score
contributions.

<img width="3009" height="1575" alt="Screenshot 2026-04-02 at 17 36 57"
src="https://github.com/user-attachments/assets/5437d8a3-4f63-4d39-8aa8-4915bc9cff91"
/>

- Updates the entity flyout risk summary to show entity and
resolution-group scoring coherently, including corrected score sourcing
and filtering behavior.
- Reworks the risk contributions tab for resolution scoring by replacing
the separate contributing-entities section with inline attribution in
alerts and contexts.
- Uses entity-store-driven attribution for resolution context rows
(watchlists and criticality) and alert-derived attribution for
contributing alerts.
- Reduces flyout refresh noise by limiting loading indicators to
initial/empty states and adjusting table layout for readability.
- Changes attribution display to Entity ID and links host/user IDs to
preview-mode entity flyouts.

## Testing

### Feature flags

Add to `kibana.yml` (or `kibana.dev.yml`):

```yaml
xpack.securitySolution.enableExperimental:
  - entityAnalyticsEntityStoreV2
  - entityAnalyticsWatchlistEnabled
```

Then enable Entity Store V2 and the ID-based risk scoring advanced
setting (`securitySolution:entityStoreEnableV2`) via Stack Management ->
Advanced Settings.

### Manual testing with the document generator

The
[security-documents-generator](https://github.com/elastic/security-documents-generator)
has a companion PR
([elastic#342](elastic/security-documents-generator#342))
with a `risk-score-v2` command that seeds entities, ingests alerts,
triggers maintainer runs, and prints a scorecard:

```bash
# In the security-documents-generator repo, on the risk-scoring-v2 branch:
yarn start risk-score-v2
```

The command supports interactive follow-on actions (expand entities,
tweak scores, view risk docs, run comparisons) and includes graph
workflows for resolution + propagation.

### What to verify

- In the entity flyout risk summary, the top card/table show the entity
(non-resolution) risk score and the card label reads `Entity risk score`
- The resolution card/table appears only when a resolution score exists
and is labeled `Resolution group risk score`
- Resolution score history is loaded using the resolution target
`id_value` and `score_type: "resolution"`
- In the Risk contributions tab, the entity/resolution toggle appears
only when a resolution score exists
- The separate contributing-entities section is removed; attribution is
shown inline in table columns
- In resolution view, both Contexts and Alerts tables show an `Entity
ID` column
- For context rows, watchlist and criticality attribution can show
multiple entity IDs when multiple entities match
- Watchlist rows display watchlist names (resolved from watchlist API
data), not only IDs
- Host/user entity IDs in attribution columns open preview-mode entity
flyouts (without replacing the current flyout)
- During background refreshes, existing table rows stay visible and
loading indicators appear only for initial/empty states

### Automated checks run

- `node scripts/jest
x-pack/solutions/security/plugins/security_solution/public/entity_analytics/components/entity_details_flyout/tabs/risk_inputs/risk_inputs.test.tsx`
- `node scripts/check_changes.ts`

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: kubasobon <kuba.sobon@elastic.co>
Co-authored-by: Kuba Soboń <wtty.fool@gmail.com>
Co-authored-by: Tiago Vila Verde <tiago.vilaverde@elastic.co>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: CAWilson94 <charlotte.wilson@elastic.co>
Co-authored-by: jaredburgettelastic <jared.burgett@elastic.co>
Co-authored-by: Jared Burgett <147995946+jaredburgettelastic@users.noreply.github.com>